Sensing user input at display area edge

ABSTRACT

One or more sensors are disposed to sense user inputs in an active display area as well as user inputs in an extended area that is outside of the active display area. Data for user inputs, such as gestures, may include data from user inputs sensed in both the active display area and outside of the active display area. The user inputs can begin and/or end outside of the active display area.

RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 13/651,195, filed Oct. 12, 2012, entitled “SensingUser Input At Display Area Edge”, which is a continuation of and claimspriority to U.S. patent application Ser. No. 13/471,376, filed May 14,2012, entitled “Sensing User Input At Display Area Edge”, and whichfurther claims priority under 35 U.S.C. §119(e) to the following U.S.Provisional patent applications, the entire disclosures of each of theseapplications being incorporated by reference in their entirety:

U.S. Provisional Patent Application No. 61/606,321, filed Mar. 2, 2012,and titled “Screen Edge;”

U.S. Provisional Patent Application No. 61/606,301, filed Mar. 2, 2012,and titled “Input Device Functionality;”

U.S. Provisional Patent Application No. 61/606,313, filed Mar. 2, 2012,and titled “Functional Hinge;”

U.S. Provisional Patent Application No. 61/606,333, filed Mar. 2, 2012,and titled “Usage and Authentication;”

U.S. Provisional Patent Application No. 61/613,745, filed Mar. 21, 2012,and titled “Usage and Authentication;”

U.S. Provisional Patent Application No. 61/606,336, filed Mar. 2, 2012,and titled “Kickstand and Camera;” and

U.S. Provisional Patent Application No. 61/607,451, filed Mar. 6, 2012,and titled “Spanaway Provisional.”

BACKGROUND

Mobile computing devices have been developed to increase thefunctionality that is made available to users in a mobile setting. Forexample, a user may interact with a mobile phone, tablet computer, orother mobile computing device to check email, surf the web, composetexts, interact with applications, and so on. Traditional mobilecomputing devices oftentimes employ displays with touchscreenfunctionality to allow users to input various data or requests to thecomputing device. However, it can be difficult to recognize certain userinputs with such traditional mobile computing devices, providingfrustrating and unfriendly experiences for the users.

SUMMARY

Sensing user input at display area edge techniques are described.

In one or more implementations, input data for a user input is received,the input data having been sensed by one or more sensors. The input dataincludes data for locations touched by an object in an active displayarea of an interactive display device as well as data for locationstouched by the object in an area outside of the active display area ofthe interactive display device. Based on the data for the locationstouched by the object in the active display area as well as thelocations touched by the object in the area outside of the activedisplay area, the user input is determined.

In one or more implementations, a computing device includes a housingand a display device supported by the housing and having an activedisplay area. The display device has one or more sensors disposed forsensing locations of the active display area that are touched by anobject while inputting a user input to the computing device as well aslocations outside of the active display area that are touched by theobject while inputting the user input. The computing device determinesthe user input based on both the locations of the active display areathat are touched by the object and the locations outside of the activedisplay area that are touched by the object.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different instances in thedescription and the figures may indicate similar or identical items.Entities represented in the figures may be indicative of one or moreentities and thus reference may be made interchangeably to single orplural forms of the entities in the discussion.

FIG. 1 is an illustration of an environment in an example implementationthat is operable to employ the techniques described herein.

FIG. 2 is an illustration of an environment in another exampleimplementation that is operable to employ the techniques describedherein.

FIG. 3 depicts an example implementation of an input device of FIG. 2 asshowing a flexible hinge in greater detail.

FIG. 4 depicts an example implementation showing a perspective view of aconnecting portion of FIG. 3 that includes mechanical couplingprotrusions and a plurality of communication contacts.

FIG. 5 illustrates an example display device implementing the sensinguser input at display area techniques.

FIG. 6 illustrates a cross section view of an example display deviceimplementing the sensing user input at display area techniques.

FIG. 7 illustrates a cross section view of another example displaydevice implementing the sensing user input at display area techniques.

FIG. 8 is an illustration of a system in an example implementation thatis operable to employ the techniques described herein.

FIG. 9 illustrates the example display device of FIG. 5 with an exampleuser input.

FIG. 10 illustrates the example display device of FIG. 5 with anotherexample user input.

FIG. 11 illustrates the example display device of FIG. 5 with anotherexample user input.

FIG. 12 illustrates the example display device of FIG. 5 with anotherexample user input.

FIG. 13 is a flowchart illustrating an example process for implementingthe techniques described herein in accordance with one or moreembodiments.

FIG. 14 illustrates an example system including various components of anexample device that can be implemented as any type of computing deviceas described with reference to FIGS. 1-13 to implement embodiments ofthe techniques described herein.

DETAILED DESCRIPTION Overview

Sensing user input at display area edge techniques are described. One ormore sensors are disposed to sense user inputs in an active display areaas well as to sense user inputs in an extended area that is outside ofthe active display area. Data for user inputs, such as gestures, mayinclude data from user inputs sensed in both the active display area andoutside of the active display area. Thus, user inputs can begin and/orend outside of the active display area.

In the following discussion, an example environment is first describedthat may employ the techniques described herein. Example procedures arethen described which may be performed in the example environment as wellas other environments. Consequently, performance of the exampleprocedures is not limited to the example environment and the exampleenvironment is not limited to performance of the example procedures.

Example Environment and Procedures

FIG. 1 is an illustration of an environment 100 in an exampleimplementation that is operable to employ the techniques describedherein. The illustrated environment 100 includes an example of acomputing device 102, which may be configured in a variety of ways. Forexample, the computing device 102 may be configured for mobile use, suchas a mobile phone, a tablet computer, and so on. However, the techniquesdiscussed herein are also applicable to multiple types of devices otherthan those for mobile use, and may be used with any of a variety ofdifferent devices that use an input sensor over or in a display area.For example, the computing device 102 may be a desktop computer, a pointof sale kiosk, an interactive display or monitor (e.g., in a hospital,airport, mall, etc.), and so forth. The computing device 102 may rangefrom full resource devices with substantial memory and processorresources to a low-resource device with limited memory and/or processingresources. The computing device 102 may also relate to software thatcauses the computing device 102 to perform one or more operations.

The computing device 102, for instance, is illustrated as including aninput/output module 108. The input/output module 108 is representativeof functionality relating to processing of inputs and rendering outputsof the computing device 102. A variety of different inputs may beprocessed by the input/output module 108, such as inputs relating tofunctions that correspond to keys of an input device coupled tocomputing device 102 or keys of a virtual keyboard displayed by thedisplay device 110, inputs that are gestures recognized throughtouchscreen functionality of the display device 110 and that causeoperations to be performed that correspond to the gestures, and soforth. The display device 110 is thus also referred to as an interactivedisplay device due to the ability of the display device to receive userinputs via any of various input sensing technologies. The input/outputmodule 108 may support a variety of different input techniques byrecognizing and leveraging a division between types of inputs includingkey presses, gestures, and so on.

FIG. 2 is an illustration of an environment 200 in another exampleimplementation that is operable to employ the techniques describedherein. The illustrated environment 200 includes an example of acomputing device 202 that is physically and communicatively coupled toan input device 204 via a flexible hinge 206. The computing device 202may be configured in a variety of ways, analogous to computing device102 of FIG. 1. The computing device 202 may also relate to software thatcauses the computing device 202 to perform one or more operations.

The computing device 202, for instance, is illustrated as including aninput/output module 208. The input/output module 208 is representativeof functionality relating to processing of inputs and rendering outputsof the computing device 202. A variety of different inputs may beprocessed by the input/output module 208, such as inputs relating tofunctions that correspond to keys of the input device 204 or keys of avirtual keyboard displayed by the display device 210, inputs that aregestures recognized through touchscreen functionality of the displaydevice 210 and that cause operations to be performed that correspond tothe gestures, and so forth. The display device 210 is thus also referredto as an interactive display device due to the ability of the displaydevice to receive user inputs via any of various input sensingtechnologies. The input/output module 208 may support a variety ofdifferent input techniques by recognizing and leveraging a divisionbetween types of inputs including key presses, gestures, and so on.

In the illustrated example, the input device 204 is configured as akeyboard having a QWERTY arrangement of keys although other arrangementsof keys are also contemplated. Further, other non-conventionalconfigurations are also contemplated, such as a game controller,configuration to mimic a musical instrument, and so forth. Thus, theinput device 204 and keys incorporated by the input device 204 mayassume a variety of different configurations to support a variety ofdifferent functionality.

As previously described, the input device 204 is physically andcommunicatively coupled to the computing device 202 in this examplethrough use of a flexible hinge 206. The flexible hinge 206 is flexiblein that rotational movement supported by the hinge is achieved throughflexing (e.g., bending) of the material forming the hinge as opposed tomechanical rotation as supported by a pin, although that embodiment isalso contemplated. Further, this flexible rotation may be configured tosupport movement in one direction (e.g., vertically in the figure) yetrestrict movement in other directions, such as lateral movement of theinput device 204 in relation to the computing device 202. This may beused to support consistent alignment of the input device 204 in relationto the computing device 202, such as to align sensors used to changepower states, application states, and so on.

The flexible hinge 206, for instance, may be formed using one or morelayers of fabric and include conductors formed as flexible traces tocommunicatively couple the input device 204 to the computing device 202and vice versa. This communication, for instance, may be used tocommunicate a result of a key press to the computing device 202, receivepower from the computing device, perform authentication, providesupplemental power to the computing device 202, and so on. The flexiblehinge 206 may be configured in a variety of ways, further discussion ofwhich may be found in relation to the following figure.

FIG. 3 depicts an example implementation 300 of the input device 204 ofFIG. 2 as showing the flexible hinge 206 in greater detail. In thisexample, a connection portion 302 of the input device is shown that isconfigured to provide a communicative and physical connection betweenthe input device 204 and the computing device 202. In this example, theconnection portion 302 has a height and cross section configured to bereceived in a channel in the housing of the computing device 202,although this arrangement may also be reversed without departing fromthe spirit and scope thereof.

The connection portion 302 is flexibly connected to a portion of theinput device 204 that includes the keys through use of the flexiblehinge 206. Thus, when the connection portion 302 is physically connectedto the computing device the combination of the connection portion 302and the flexible hinge 206 supports movement of the input device 204 inrelation to the computing device 202 that is similar to a hinge of abook.

For example, rotational movement may be supported by the flexible hinge206 such that the input device 204 may be placed against the displaydevice 210 of the computing device 202 and thereby act as a cover. Theinput device 204 may also be rotated so as to be disposed against a backof the computing device 202, e.g., against a rear housing of thecomputing device 202 that is disposed opposite the display device 210 onthe computing device 202.

Naturally, a variety of other orientations are also supported. Forinstance, the computing device 202 and input device 204 may assume anarrangement such that both are laid flat against a surface as shown inFIG. 2. In another instance, a typing arrangement may be supported inwhich the input device 204 is laid flat against a surface and thecomputing device 202 is disposed at an angle to permit viewing of thedisplay device 210, e.g., such as through use of a kickstand disposed ona rear surface of the computing device 202. Other instances are alsocontemplated, such as a tripod arrangement, meeting arrangement,presentation arrangement, and so forth.

The connecting portion 302 is illustrated in this example as includingmagnetic coupling devices 304, 306, mechanical coupling protrusions 308,310, and a plurality of communication contacts 312. The magneticcoupling devices 304, 306 are configured to magnetically couple tocomplementary magnetic coupling devices of the computing device 202through use of one or more magnets. In this way, the input device 204may be physically secured to the computing device 202 through use ofmagnetic attraction.

The connecting portion 302 also includes mechanical coupling protrusions308, 310 to form a mechanical physical connection between the inputdevice 204 and the computing device 202. The mechanical couplingprotrusions 308, 310 are shown in greater detail in the followingfigure.

FIG. 4 depicts an example implementation 400 showing a perspective viewof the connecting portion 302 of FIG. 3 that includes the mechanicalcoupling protrusions 308, 310 and the plurality of communicationcontacts 312. As illustrated, the mechanical coupling protrusions 308,310 are configured to extend away from a surface of the connectingportion 302, which in this case is perpendicular although other anglesare also contemplated.

The mechanical coupling protrusions 308, 310 are configured to bereceived within complimentary cavities within the channel of thecomputing device 202. When so received, the mechanical couplingprotrusions 308, 310 promote a mechanical binding between the deviceswhen forces are applied that are not aligned with an axis that isdefined as correspond to the height of the protrusions and the depth ofthe cavity.

For example, when a force is applied that does coincide with thelongitudinal axis described previously that follows the height of theprotrusions and the depth of the cavities, a user overcomes the forceapplied by the magnets solely to separate the input device 204 from thecomputing device 202. However, at other angles the mechanical couplingprotrusion 308, 310 are configured to mechanically bind within thecavities, thereby creating a force to resist removal of the input device204 from the computing device 202 in addition to the magnetic force ofthe magnetic coupling devices 304, 306. In this way, the mechanicalcoupling protrusions 308, 310 may bias the removal of the input device204 from the computing device 202 to mimic tearing a page from a bookand restrict other attempts to separate the devices.

The connecting portion 302 is also illustrated as including a pluralityof communication contacts 312. The plurality of communication contacts312 is configured to contact corresponding communication contacts of thecomputing device 202 to form a communicative coupling between thedevices. The communication contacts 312 may be configured in a varietyof ways, such as through formation using a plurality of spring loadedpins that are configured to provide a consistent communication contactbetween the input device 204 and the computing device 202. Therefore,the communication contact may be configured to remain during minormovement of jostling of the devices. A variety of other examples arealso contemplated, including placement of the pins on the computingdevice 202 and contacts on the input device 204.

The sensing user input at display area edge techniques use one or moresensors disposed in an extended sensor area to sense user input outsideof an active display area. One or more sensors are also disposed tosense user inputs in the active display area. The extended sensor areais in close proximity to (e.g., within 5 millimeters of) the activedisplay area, and typically is adjacent to the active display area.

FIG. 5 illustrates an example display device 500 implementing thesensing user input at display area techniques. The display device 500 isan interactive display device that includes an active display area 502in which various data and information may be displayed by the computingdevice. The display area 502 is referred to as an active display area asthe data and information displayed can be changed over time by thecomputing device, optionally in response to user inputs received by thecomputing device. The display device 500 also includes an extendedsensor area 504, surrounding and adjacent to the active display area502, illustrated with cross-hatching. User inputs can be received whenan object, such as a finger of a user's hand, a stylus, a pen, and soforth is touching and/or in close proximity to the surface of the activedisplay area 502 and/or the surface of the extended sensor area 504.Extended sensor area 504 facilitates sensing user inputs along the edgeof the active display area 502. The edge of the active display area 502refers to the outer perimeter of the active display area 502, which isthe portion of the active display area 502 that is closest to theextended sensor area 504.

The extended sensor area 504 can extend, for example, 2 millimetersbeyond the active display area 502, although other amounts of extensionare contemplated. The extended sensor area 504 can extend the sameamount beyond the active display area 502 all around the active displayarea 502, or alternatively can extend by different amounts. For example,the extended sensor area 504 can extend beyond the active display area502 by 2 millimeters in the vertical direction and by 4 millimeters inthe horizontal direction. The extended sensor area 504 can also vary fordifferent types of devices and be customized to the particular type ofdevice. For example, interactive devices that can receive input fromfarther away (e.g., point of sale kiosks and interactive displays thatcan sense input as far away as 10 centimeters) may have extended sensorareas that extend beyond the display area farther (e.g., 10-15centimeters rather than 2-4 millimeters) than devices that receive inputfrom closer interactions (e.g., a tablet that senses touch).

Display devices implementing the sensing user input at display area edgetechniques can use a variety of active display technologies. Theseactive display technologies may include, for example, flexible displaytechnologies, e-reader display technologies, liquid crystal (LCD)display technologies, light-emitting diode (LED) display technologies,organic light-emitting diode (OLED) display technologies, plasma displaytechnologies, and so forth. Although examples of display technologiesare discussed herein, other display technologies are also contemplated.

Display devices implementing the sensing user input at display area edgetechniques can use a variety of different input sensing technologies.These input sensing technologies may include capacitive systems and/orresistive systems that sense touch. These input sensing technologies mayalso include inductive systems that sense pen (or other object) inputs.These input sensing technologies may also include optical based systemsthat sense reflection or disruption of light from objects touching (orclose to) the surface of the display device, such as Sensor in Pixel(SIP) systems, Infrared systems, optical imaging systems, and so forth.Other types of input sensing technologies can also be used, such assurface acoustic wave systems, acoustic pulse recognition systems,dispersive signal systems, and so forth. Although examples of inputsensing technologies are discussed herein, other input sensingtechnologies are also contemplated. Furthermore, these input sensingtechnologies may be combined together, such as a piezoelectric withextended capacitive sensor to provide other tactile input.

Depending on the input sensing technology that is used for a displaydevice, user inputs can be received when an object (such as a finger ofa user's hand, a stylus, a pen, and so forth) is touching and/or inclose proximity to the surface of the display device. This closeproximity can be, for example 5 millimeters, although differentproximities are contemplated and can vary depending on the manner inwhich the display device is implemented. The proximity of an object tothe display device refers to a distance the object is from the displaydevice along a direction perpendicular to a plane of the display device.

FIG. 6 illustrates a cross section view of an example display device 600implementing the sensing user input at display area techniques. Thedisplay device 600 includes an active display layer 602 on top of whichis disposed an input sensing layer 604. Although the layers 602 and 604are illustrated as being individual layers, it should be noted that eachof the layers 602 and 604 itself may be made up of multiple layers. Theinput sensing layer 604 and the active display layer 602 can beimplemented using a variety of different technologies, as discussedabove. Although not illustrated in FIG. 6, it should be noted that anynumber of additional layers can be included in the display device 600.For example, an additional protective layer made of glass or plastic canbe disposed on top of input sensing layer 604.

A user's finger 606 (or other object) touching or in close proximity tothe input sensing layer 604 is sensed by the input sensing layer 604.The locations where the user's finger 606 (or other object) is sensed bythe layer 604 is provided by the layer 604 as sensed object locationsand are used to identify the user input, as discussed in more detailbelow.

The input sensing layer 604 includes multiple sensors, and extendsbeyond the active display area 602 to extended sensor area 608, 610. Thenumber of sensors and manner in which the sensors are disposed may varybased on the implementation and the input sensing technology used forthe input sensing layer 604. The input sensing layer 604 includes aportion 612 as well as portions 614 and 616.

One or more sensors may be disposed in the input sensing layer 604 aboveactive display layer 602, in portion 612. These sensors disposed abovethe layer 602 sense the user's finger 606 (or other object) touching orin close proximity to the layer 604 above the active display layer 602,and thus are also referred to as sensing user input in and/or above theactive display area as well as being disposed in the active displayarea.

One or more sensors may also be disposed in the input sensing layer 604above extended sensor area 608, 610, in portions 614, 616, respectively.The extended sensor area 608, 610 is not above the active display layer602, as illustrated in FIG. 6. These sensors disposed above the extendedsensor area 608, 610 sense the user's finger 606 (or other object)touching or in close proximity to the layer 604 above the extendedsensor area 608, 610, and thus are also referred to as sensing userinput in and/or above the extended sensor area 608, 610. Because theextended sensor area 608, 610 is not above the active display layer 602,these sensors disposed above the extended sensor area 608, 610 are alsoreferred to as sensing user input in an area outside of the activedisplay area as well as being disposed in the area outside of the activedisplay area.

Alternatively, sensors may be disposed in the input sensing layer 604 inother manners, such as along the outer edge (the perimeter) of the inputsensing layer 604, at corners of the input sensing layer 604, and soforth. Such sensors may still sense user input in and/or above theactive display area, as well as user input in an area outside of theactive display area.

FIG. 7 illustrates a cross section view of another example displaydevice 700 implementing the sensing user input at display areatechniques. The display device 700 includes an active display layer 702on top of which is disposed an input sensing layer 704. The inputsensing layer 704 and the active display layer 702 can be implementedusing a variety of different technologies, as discussed above. Thelayers 702 and 704 are disposed between a lower panel layer 706 and anupper panel layer 708. The panel layers 706, 708 may be made of variousmaterials, such as glass, plastic, and so forth. Although the layers702, 704, 706, and 708 are illustrated as being individual layers, itshould be noted that each of the layers 702, 704, 706, and 708 itselfmay be made up of multiple layers. These layers may also be flexiblelayers, and applicable in 3-dimensional (3Ds) interactive devices.

Additional support material 714, 716 is optionally included between thepanel layers 706, 708, illustrated with cross-hatching in FIG. 7. Thesupport material 714, 716 provides additional support for areas betweenthe panel layers to which the layers 702 and 704 do not extend. Thesupport material 714, 716 can be various materials, such as glass,plastic, bonding adhesive, and so forth.

A user's finger 606 (or other object) touching or in close proximity tothe input sensing layer 704 is sensed by the input sensing layer 704.The locations where the user's finger 606 (or other object) is sensed bythe layer 704 is provided by the layer 704 as sensed object locationsand are used to identify the user input, as discussed in more detailbelow.

The input sensing layer 704 includes multiple sensors, and extendsbeyond the active display area 702 to extended sensor area 710, 712.Input sensing layer 704 need not, however, extend as far as panel layers706, 708, as illustrated. The number of sensors included in the inputsensing layer 704 and the manner in which the sensors are disposed mayvary based on the implementation and the input sensing technology usedfor the input sensing layer 704. The input sensing layer 704 includes aportion 718 as well as portions 720 and 722.

One or more sensors are disposed in the input sensing layer 704 aboveactive display layer 702, in portion 718. These sensors disposed abovethe layer 702 sense the user's finger 606 (or other object) touching orin close proximity to the panel layer 708 above the active display layer702, and thus are also referred to as sensing user input in and/or abovethe active display area as well as being disposed in the active displayarea.

One or more sensors are also disposed in the input sensing layer 704above extended sensor area 710, 712, in portions 720, 722, respectively.The extended sensor area 710, 712 is not above the active display layer702, as illustrated in FIG. 7. These sensors disposed above the extendedsensor area 710, 712 sense the user's finger 706 (or other object)touching or in close proximity to the panel layer 708 above the extendedsensor area 710, 712, and thus are also referred to as sensing userinput in and/or above the extended sensor area 710, 712. Because theextended sensor area 710, 712 is not above the active display layer 702,these sensors disposed above the extended sensor area 710, 712 are alsoreferred to as sensing user input in an area outside of the activedisplay area as well as being disposed in the area outside of the activedisplay area.

Alternatively, sensors may be disposed in the input sensing layer 704 inother manners, such as along the outer edge (the perimeter) of the inputsensing layer 704, at corners of the input sensing layer 704, and soforth. Such sensors may still sense user input in and/or above theactive display area, as well as user input in an area outside of theactive display area.

It should be noted that, although the input sensing layers in FIGS. 6and 7 are illustrated as being disposed above the active display layers,other arrangements are contemplated. For example, the input sensinglayer can be within or below the active display layer. The input sensinglayer can also be of multiple configurations. The input sensing layermay be on both sides of plastic and/or glass substrate, or on the sameside of plastic, glass and/or other optical clear layers.

FIG. 8 is an illustration of a system 800 in an example implementationthat is operable to employ the techniques described herein. The system800 includes an input data collection module 802 and an input handlermodule 804. System 800 may be implemented, for example, in the computingdevice 102 of FIG. 1 or the computing device 202 of FIG. 2. Although themodules 802 and 804 are illustrated in the system 800, it should benoted that one or more additional modules may be included in the system800. It should also be noted that the functionality of the module 802and/or the module 804 can be separated into multiple modules.

The input data collection module 802 receives indications of sensedobject locations 806. These sensed object location indications 806 areindications of locations of an object (e.g., the user's finger or a pen)that were sensed by an input sensing layer of a display device. Timinginformation associated with the locations that were sensed by the inputsensing layer can also optionally be included as part of the sensedobject location indications 806. This timing information indicates whena particular location was sensed, and may take different forms. Forexample, this timing information may be relative to a fixed timeframe orclock, or may be an amount of time since the previous location wassensed. Alternatively, the timing information may be generated by theinput data collection module 802 based on the timing of receipt of thesensed object location indications 806.

The input data collection module 802 uses the sensed object locationindications 806 to generate input data 808. The input data 808 describesthe location and the movement of the user input. The input data 808 canbe the sensed object location indications 806, as well as any associatedtiming information for the locations as received and/or generated by themodule 802.

Additionally, a user input can have an associated lifetime, which refersto a time duration that begins when an object touching (or in closeproximity to) the surface is sensed and ends when the object is nolonger sensed as touching (or in close proximity to) the surface of thedisplay device. This associated lifetime may be identified by the inputdata collection module 802 and included as part of the input data 808.

A user input can also have an associated velocity, which refers to avelocity at which the object that is sensed is moving. This velocity isa particular distance divided by a particular amount of time, such as aparticular number of inches per second, a particular number ofmillimeters per millisecond, and so forth. This associated velocity maybe identified by the input data collection module 802 and included aspart of the input data 808, or used in other manners (e.g., to determinewhen to provide input data 808 to the input handler module 804, asdiscussed in more detail below).

The input data collection module 802 provides the input data 808 to theinput handler module 804, which determines what the user input is. Theuser input can take various forms, such as a gesture or mouse movement.A gesture refers to a motion or path taken by an object (e.g., theuser's finger) to initiate one or more functions of a computing device.For example, a gesture may be sliding of the user's finger in aparticular direction, the user's finger tracing a particular characteror symbol (e.g., a circle, a letter “Z”, etc.), and so forth. A gesturemay also include a multi-touch input in which multiple objects (e.g.,multiple of the user's fingers) take particular motions or paths toinitiate one or more functions of the computing device. A mouse movementrefers to a motion or path taken by an object (e.g., the user's finger)to move something (e.g., a cursor or pointer, an object being draggedand dropped, etc.) on the display device. Although gestures and mousemovements are discussed herein, various other types of user inputs arecontemplated.

The input handler module 804 may use any of a variety of public and/orproprietary techniques to determine what the user input is based on theinput data 808. For example, the input handler module 804 can determinethat the user input is a particular gesture, a particular mousemovement, and so forth. The input handler 804 may also be configured toanalyze characteristics of the input (e.g., the size of the input and/orvelocity of the input) to configure the display or other output for acustomized user experience. For example, a small finger with small inputcan be processed to adjust the font, color, application, and so forthsuitable to children.

The input handler module 804 may also, based on the determined userinput, take various actions. For example, the input handler module 804may provide an indication of the determined user input to one or moreother modules of the computing device to carry out the requestedfunction or movement. By way of another example, the input handlermodule 804 itself may carry out the requested function or movement.

The input data collection module 802 may provide the input data 808 tothe input handler module 804 at various times. For example, the inputdata collection module 802 may provide the input data 808 to the inputhandler module 804 as the input data 808 is generated. By way of anotherexample, the input data collection module 802 may provide the input data808 to the input handler after the user input has finished (e.g., afterthe lifetime associated with the user input has elapsed and the objectis no longer sensed as touching (or in close proximity to) the surfaceof the display device).

Alternatively, the input data collection module 802 may maintain theinput data 808 for a user input but not provide the input data 808 tothe input handler module 804 until a particular event occurs. Variousdifferent events can cause the module 802 to provide the input data 808to the module 804. One event that may cause the module 802 to providethe input data 808 to the module 804 is the user input, as indicated bythe location of the object, being in the active display area. Thus, inresponse to the user input being in the active display area, the module802 provides the input data 808 to the module 804.

Another event that may cause the module 802 to provide the input data808 to the module 804 is the user input being outside of the activedisplay area but predicted to be in the active display area in thefuture (e.g., during an associated lifetime of the user input). The userinput can be predicted to be in the active display area in the futurebased on various rules or criteria, such as based on the velocity of theuser input and/or the direction of the user input. For example, if theuser input is outside of the active display area and the direction ofthe user input is towards the active display area, then the user inputis predicted to be in the active display area in the future. By way ofanother example, if the user input is outside of the active displayarea, the direction of the user input is towards the active displayarea, and the velocity of the user input is greater than a thresholdamount, then the user input is predicted to be in the active displayarea in the future. This threshold amount can be, for example, 4 inchesper second, although other threshold amounts are contemplated. Thus, inresponse to the user input being predicted to be in the active displayarea in the future, the module 802 provides the input data 808 to themodule 804.

FIG. 9 illustrates the example display device 500 of FIG. 5 with anexample user input. The display device 500 includes an active displayarea 502 surrounded by an extended sensor area 504, illustrated withcross-hatching, as discussed above. A user input is received via auser's finger 606.

The user input in FIG. 9 is illustrated as a movement from right toleft, with the user input beginning in the extended sensor area 504 andmoving into the active display area 502. The ending position of theuser's finger is illustrated using a dashed outline of a hand. Sensingof the user input begins in the extended sensor area 504, prior to theuser's finger 606 moving into the active display area 502. The userinput indicated by the movement of the user's finger 606 in FIG. 9 maybe identified more quickly than if extended sensor area 504 were notincluded in display device 500. The user input may be identified morequickly because without extended sensor area 504 locations of the user'sfinger 606 would not begin to be sensed until after the edge of theactive display area 502 is reached by the user's finger 606.

The user input in FIG. 9 is illustrated as beginning in extended sensorarea 504. However, it should be noted that the user input can beginoutside of both the active display area 502 and the extended sensor area504 (e.g., along an edge of the display device 500). The user input maystill be identified more quickly than if extended sensor area 504 werenot included in display device 502 because the movement will begin to besensed when the extended sensor area 504 is reached by the user's finger606 (rather than waiting until the user's finger 606 reaches the activedisplay area 502).

FIG. 10 illustrates the example display device 500 of FIG. 5 withanother example user input. The display device 500 includes an activedisplay area 502 surrounded by an extended sensor area 504, illustratedwith cross-hatching, as discussed above. A user input is received via auser's finger 606.

The user input in FIG. 10 is illustrated as a movement from left toright, with the user input beginning in the active display area 502 andending in the extended sensor area 504. The ending position of theuser's finger is illustrated using a dashed outline of a hand.Alternatively, the ending position of the movement may be outside ofboth the active display area 502 and the extended sensor area 504 (e.g.,along an edge of the display device 500). Sensing of the user inputbegins in the active display area, prior to the user's finger 606 movinginto the extended sensor area 504. By ending movement of the user'sfinger 606 in (or having the movement of the user's finger pass through)the extended sensor area 504, the location of the user input in theextended sensor area 504 can be used in identifying the user input. Forexample, the input handler module 804 of FIG. 8 may determine that theuser input is a swipe or gesture from left to right across the displaydevice as opposed to an input that was intended by the user to stop overa particular icon or object displayed near the edge of the display area.

FIG. 11 illustrates the example display device 500 of FIG. 5 withanother example user input. The display device 500 includes an activedisplay area 502 surrounded by an extended sensor area 504, illustratedwith cross-hatching, as discussed above. A user input is received via auser's finger 606.

The user input in FIG. 11 is illustrated as moving from right to leftand from top to bottom in a “<” shape. The user input in FIG. 11 beginsand ends in the active display area 504, but passes through the extendedsensor area 504. The ending position of the user's finger is illustratedusing a dashed outline of a hand. Sensing of the user input in theextended sensor area 504 allows the user input illustrated in FIG. 11 tobe input along the edge of the active display area 504. Even though theuser input passes outside the edge of the active display area 504, theuser input is sensed in the extended sensor area 504.

FIG. 12 illustrates the example display device 500 of FIG. 5 withanother example user input. The display device 500 includes an activedisplay area 502 surrounded by an extended sensor area 504, illustratedwith cross-hatching, as discussed above. A user input is received via auser's finger 606.

The user input in FIG. 12 is illustrated as a movement from left toright, with the user input beginning and ending in the extended sensorarea 504 without moving into the active display area 502. The endingposition of the user's finger is illustrated using a dashed outline of ahand. Sensing of the user input begins in the extended sensor area 504.However, as the user's finger 606 is not moved into the active displayarea 502, and the direction of movement of the user's finger 606 is nottowards the active display area 502, the input data for the user inputneed not be provided to the input handler module 804 of FIG. 8. Thus, asthe user input remains in the extended sensor area 504, no action basedon a user input need be taken.

FIG. 13 is a flowchart illustrating an example process 1300 forimplementing the techniques described herein in accordance with one ormore embodiments. Process 1300 is carried out by a computing device,such as computing device 102 of FIG. 1 or computing device 202 of FIG.2, and can be implemented in software, firmware, hardware, orcombinations thereof. Process 1300 is shown as a set of acts and is notlimited to the order shown for performing the operations of the variousacts. Process 1300 is an example process for implementing the techniquesdescribed herein; additional discussions of implementing the techniquesdescribed herein are included herein with reference to differentfigures.

In process 1300, input data is received (act 1302). The input dataincludes data for at least part of the user input in an active displayarea of a device and data for at least part of the user input in an areaoutside of the active display area of the device, as discussed above.

Based on the input data, the user input is determined (act 1304). Any ofa variety of public and/or proprietary techniques may be used todetermine what the user input is, as discussed above.

The action indicated by the user input is performed (act 1306). Thisaction may be the performance of various functions or movements, asdiscussed above.

Example System and Device

FIG. 14 illustrates an example system generally at 1400 that includes anexample computing device 1402 that is representative of one or morecomputing systems and/or devices that may implement the varioustechniques described herein. The computing device 1402 may, for example,be configured to assume a mobile configuration through use of a housingformed and sized to be grasped and carried by one or more hands of auser, illustrated examples of which include a mobile phone, mobile gameand music device, and tablet computer although other examples andconfigurations are also contemplated.

The example computing device 1402 as illustrated includes a processingsystem 1404, one or more computer-readable media 1406, and one or moreI/O interfaces 1408 that are communicatively coupled, one to another.Although not shown, the computing device 1402 may further include asystem bus or other data and command transfer system that couples thevarious components, one to another. A system bus can include any one orcombination of different bus structures, such as a memory bus or memorycontroller, a peripheral bus, a universal serial bus, and/or a processoror local bus that utilizes any of a variety of bus architectures. Avariety of other examples are also contemplated, such as control anddata lines.

The processing system 1404 is representative of functionality to performone or more operations using hardware. Accordingly, the processingsystem 1404 is illustrated as including hardware element 1410 that maybe configured as processors, functional blocks, and so forth. This mayinclude implementation in hardware as an application specific integratedcircuit or other logic device formed using one or more semiconductors.The hardware elements 1410 are not limited by the materials from whichthey are formed or the processing mechanisms employed therein. Forexample, processors may be comprised of semiconductor(s) and/ortransistors (e.g., electronic integrated circuits (ICs)). In such acontext, processor-executable instructions may beelectronically-executable instructions.

The computer-readable storage media 1406 is illustrated as includingmemory/storage 1412. The memory/storage 1412 represents memory/storagecapacity associated with one or more computer-readable media. Thememory/storage component 1412 may include volatile media (such as randomaccess memory (RAM)) and/or nonvolatile media (such as read only memory(ROM), Flash memory, optical disks, magnetic disks, and so forth). Thememory/storage component 1412 may include fixed media (e.g., RAM, ROM, afixed hard drive, and so on) as well as removable media (e.g., Flashmemory, a removable hard drive, an optical disc, and so forth). Thecomputer-readable media 1406 may be configured in a variety of otherways as further described below.

Input/output interface(s) 1408 are representative of functionality toallow a user to enter commands and information to computing device 1402,and also allow information to be presented to the user and/or othercomponents or devices using various input/output devices. Examples ofinput devices include a keyboard, a cursor control device (e.g., amouse), a microphone, a scanner, touch functionality (e.g., capacitiveor other sensors that are configured to detect physical touch), a camera(e.g., which may employ visible or non-visible wavelengths such asinfrared frequencies to recognize movement as gestures that do notinvolve touch), and so forth. Examples of output devices include adisplay device (e.g., a monitor or projector), speakers, a printer, anetwork card, tactile-response device, and so forth. Thus, the computingdevice 1402 may be configured in a variety of ways to support userinteraction.

The computing device 1402 is further illustrated as including one ormore modules 1418 that may be configured to support a variety offunctionality. The one or more modules 1418, for instance, may beconfigured to generate input data based on indications of sensed objectlocations, to determine what a user input is based on the input data,and so forth. The modules 1418 may include, for example, the input datacollection module 802 and/or the input handler module 804 of FIG. 8.

Various techniques may be described herein in the general context ofsoftware, hardware elements, or program modules. Generally, such modulesinclude routines, programs, objects, elements, components, datastructures, and so forth that perform particular tasks or implementparticular abstract data types. The terms “module,” “functionality,” and“component” as used herein generally represent software, firmware,hardware, or a combination thereof. The features of the techniquesdescribed herein are platform-independent, meaning that the techniquesmay be implemented on a variety of commercial computing platforms havinga variety of processors.

An implementation of the described modules and techniques may be storedon or transmitted across some form of computer-readable media. Thecomputer-readable media may include a variety of media that may beaccessed by the computing device 1402. By way of example, and notlimitation, computer-readable media may include “computer-readablestorage media” and “computer-readable signal media.”

“Computer-readable storage media” may refer to media and/or devices thatenable persistent and/or non-transitory storage of information incontrast to mere signal transmission, carrier waves, or signals per se.Thus, computer-readable storage media refers to non-signal bearingmedia. The computer-readable storage media includes hardware such asvolatile and nonvolatile, removable and non-removable media and/orstorage devices implemented in a method or technology suitable forstorage of information such as computer readable instructions, datastructures, program modules, logic elements/circuits, or other data.Examples of computer-readable storage media may include, but are notlimited to, RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, digital versatile disks (DVD) or other optical storage, harddisks, magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or other storage device, tangible media, orarticle of manufacture suitable to store the desired information andwhich may be accessed by a computer.

“Computer-readable signal media” may refer to a signal-bearing mediumthat is configured to transmit instructions to the hardware of thecomputing device 1402, such as via a network. Signal media typically mayembody computer readable instructions, data structures, program modules,or other data in a modulated data signal, such as carrier waves, datasignals, or other transport mechanism. Signal media also include anyinformation delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media include wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared, and other wireless media.

As previously described, hardware elements 1410 and computer-readablemedia 1406 are representative of modules, programmable device logicand/or fixed device logic implemented in a hardware form that may beemployed in some embodiments to implement at least some aspects of thetechniques described herein, such as to perform one or moreinstructions. Hardware may include components of an integrated circuitor on-chip system, an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), a complex programmable logicdevice (CPLD), and other implementations in silicon or other hardware.In this context, hardware may operate as a processing device thatperforms program tasks defined by instructions and/or logic embodied bythe hardware as well as a hardware utilized to store instructions forexecution, e.g., the computer-readable storage media describedpreviously.

Combinations of the foregoing may also be employed to implement varioustechniques described herein. Accordingly, software, hardware, orexecutable modules may be implemented as one or more instructions and/orlogic embodied on some form of computer-readable storage media and/or byone or more hardware elements 1410. The computing device 1402 may beconfigured to implement particular instructions and/or functionscorresponding to the software and/or hardware modules. Accordingly,implementation of a module that is executable by the computing device1402 as software may be achieved at least partially in hardware, e.g.,through use of computer-readable storage media and/or hardware elements1410 of the processing system 1404. The instructions and/or functionsmay be executable/operable by one or more articles of manufacture (forexample, one or more computing devices 1402 and/or processing systems1404) to implement techniques, modules, and examples described herein.

CONCLUSION

Although the example implementations have been described in languagespecific to structural features and/or methodological acts, it is to beunderstood that the implementations defined in the appended claims isnot necessarily limited to the specific features or acts described.Rather, the specific features and acts are disclosed as example forms ofimplementing the claimed features.

What is claimed is:
 1. A method comprising: receiving input data for auser input, the input data having been sensed by one or more sensors,the input data including data indicating first multiple locationstouched by an object in an active display area of an interactive displaydevice in which data or information is displayed as well as first timinginformation indicating when each of the first multiple locations wassensed, the input data further including data indicating second multiplelocations touched by the object in an area outside of the active displayarea of the interactive display device as well as second timinginformation indicating when each of the second multiple locations wassensed, the area outside of the active display area extending beyond theactive display area where there are no display pixels; and determiningthe user input based on the indicated first multiple locations and theindicated first timing information as well as the indicated secondmultiple locations and the indicated second timing information, the userinput comprising a gesture described by a path that includes the firstmultiple locations and the second multiple locations.
 2. A method asrecited in claim 1, the user input beginning in the area outside of theactive display area and ending in the active display area.
 3. A methodas recited in claim 1, the user input beginning in the active displayarea and ending in the area outside of the active display area.
 4. Amethod as recited in claim 1, the user input beginning in the activedisplay area and ending in the active display area, and passing throughthe area outside of the active display area after beginning the userinput and prior to ending the user input.
 5. A method as recited inclaim 1, the receiving comprising receiving the input data in responseto the user input being predicted to be in the active display area inthe future.
 6. A method as recited in claim 5, the user input beingpredicted to be in the active display area in the future in response toa direction of the user input being towards the active display area. 7.A method as recited in claim 5, the user input being predicted to be inthe active display area in the future in response to both a direction ofthe user input being towards the active display area and a velocity ofthe user input being greater than a threshold amount.
 8. A method asrecited in claim 1, the interactive display device being included in acomputing device, and the user input comprising a gesture indicating oneor more functions of the computing device to initiate.
 9. A methodcomprising: sensing, in a display device having one or more sensors,input data for a user input, the input data including first multiplelocations of an active display area of the display device in which dataor information is displayed that are touched by an object as well astiming information indicating when each of the first multiple locationswas sensed, the input data further including second multiple locationsof an area outside of the active display area that are touched by theobject as well as timing information indicating when each of the secondmultiple locations was sensed, the area outside of the active displayarea extending beyond the active display area where there is no displaylayer; and using the first multiple locations and timing informationindicating when each of the first multiple locations was sensed as wellas the second multiple locations and timing information indicating wheneach of the second multiple locations was sensed to determine the userinput.
 10. A method as recited in claim 9, the user input beginning inthe area outside of the active display area and ending in the activedisplay area.
 11. A method as recited in claim 9, the user inputbeginning in the active display area and ending in the area outside ofthe active display area.
 12. A method as recited in claim 9, the userinput beginning in the active display area and ending in the activedisplay area, and passing through the area outside of the active displayarea after beginning the user input and prior to ending the user input.13. A method as recited in claim 9, the object comprising a finger of auser providing the user input.
 14. A method as recited in claim 9, thearea outside of the active display area comprising an extended sensorarea surrounding and adjacent to the active display area.
 15. Acomputing device comprising a housing and a display device supported bythe housing and having an active display area in which data orinformation is displayed, and an area outside of the active display areaextending beyond the active display area where there are no displaypixels, the display device having one or more sensors disposed forsensing first multiple locations of the active display area that aretouched by an object while inputting a user input to the computingdevice as well as second multiple locations outside of the activedisplay area that are touched by the object while inputting the userinput, the computing device determining the user input based on both thefirst multiple locations of the active display area that are touched bythe object and the second multiple locations outside of the activedisplay area that are touched by the object, as well as based on timinginformation indicating when each of the first multiple locations wassensed by the one or more sensors and when each of the second multiplelocations was sensed by the one or more sensors.
 16. A computing deviceas recited in claim 15, at least one of the one or more sensors beingdisposed in an extended sensor area surrounding the active display areasuch that locations of the object are sensed by the computing devicealong the edge of the active display area before locations of the objectare sensed by the computing device in the active display area.
 17. Acomputing device as recited in claim 15, the one or more sensors beingincluded in an input sensor layer of the display device extending beyondan active display layer of the display device.
 18. A computing device asrecited in claim 15, the area outside of the active display areacomprising an extended sensor area surrounding and adjacent to theactive display area.
 19. A computing device as recited in claim 18, theextended sensor area extending approximately two millimeters outside theactive display area.
 20. A computing device as recited in claim 15, theuser input comprising a gesture indicating one or more functions of thecomputing device to initiate.